In recent years, a technique of realizing a three-dimensional environment on a computer, such as three-dimensional computer graphics (3DCG) or augmented reality (AR), has been actively put into practical use. In the AR technique, a virtual object or data is displayed so as to be superimposed on an object in the real world which is obtained through a camera of a portable apparatus such as a smart phone or a head mounted display (HMD). A user can visually recognize a three-dimensional image by using such a display technique. The following Patent Document 1 proposes a technique in which a user is identified and tracked within a scene by using a depth sensing camera, and avatar animation for simulating a movement of the user is displayed within the scene according to a result thereof.
However, a user interface (UI) for operating a three-dimensional environment represented by the above-described technique has been realized by using a two-dimensional input device. For example, a two-dimensional mouse operation is converted into an operation of a three-dimensional space. Thus, there are many cases where it is not easy to intuitively understand a current UI for operating a three-dimensional environment.
Therefore, the following Patent Document 2 proposes a technique in which a remote controller having a depth camera is used to detect a positional change of the remote controller, and an input command which causes an action by an application is triggered on the basis of the change. In addition, the following Patent Document 3 proposes a technique which provides a computer interaction experience to a user in a natural three-dimensional environment without the need for additional equipment such as an arm cover or a glove. In this proposal, a depth camera is installed at a position opposing a user, an image into which a virtual object is inserted is displayed along with the user captured by the depth camera, and interaction between the user and the virtual object is detected.